Aimed at the inter-block dependency, an image classification algorithm based on a two hidden Markov model(2DHMM) extension from the one dimensional HMM was developed. The 2DHMM has transition probabilities conditioned on the states of neighboring blocks from both directions. Thus, the dependency in two dimensions can be reflected simultaneously. The HMM parameters were estimated by the EM algorithm. A two dimensional version of the Viterbi algorithm was also developed to classify optimally an image based on the trained HMM. Application of the HMM algorithm to document image shows that the algorithm performs better than CART.